STOCHASTIC DERIVATIVES AND GENERALIZED /i-TRANSFORMS 

OF MARKOV PROCESSES 



CHRISTIAN LEONARD 

' Abstract. Let i? be a continuous-time Markov process on the time interval [0, 1] 

, with vakies in some state space X. We transform this reference process R into P :— 

I /o(^o) exp ^— Vt{Xt) dtj gi{Xi) R where /o,5i are nonnegative measurable func- 

tions on X and V is some measurable function on [0, 1] x X. It is easily seen that P is 
, also Markov. The aim of this paper is to identify the Markov generator of P in terms of 

lO ' the Markov generator of R and of the additional ingredients: /o, gi and V in absence of 

regularity assumptions on /o,5i and V. 

As a first step, we show that the extended generator of a Markov process is essentially 
. its stochastic derivative. Then, we compute the stochastic derivative of P to identify 

p ^ ' its generator, under a finite entropy condition. The abstract results are illustrated with 

^ , continuous diffusion processes on R'^ and Metropolis algorithms on a discrete space. 
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H ■ 1. Introduction 

■ 

We consider continuous-time Markov processes with values in some Polish space X 
equipped with its Borel cr-field. 

Notation. Let us fix some notation. The path space is the set 

n = D{[0,1],X) 

of all right continuous and left limited (cadlag) Af-valued trajectories u = {uJt)te[o,i] ^ ^• 
It is equipped with the cylindrical a-field: a{Xt;t e [0,1]) which is generated by the 
canonical process X = (Xj)jg[o_i] defined for each t G [0, 1] and u E Qhj Xt^uj) = Ut E X. 
We denote P(i^) the set of all probability measures on Q. As usual, we call process any 
P e P(f^) or any random element of Q as well. For any T G [0, 1], we denote Xj- = {Xt)teT 
and the push-forward measure Pj- = {Xf)^P. In particular, for any < r < s < 1, 
X\^r,s] = {Xt)r<t<s, P[r,s] = {X[r,s])#P and Pt = {Xt)^P G P(A') denotes the law of the 
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position Xf at time t where P{X) the set of all probability measures on X. The filtration 
is the canonical one: (cr(X[o,t]); t G [0, 1]) . 

Aim of the article. Let R be the law of some nicely behaved Markov process. We 
take this probability measure R as our reference law (this explains its unusual name R) 
and call generalized h-transform of R, any P G P(f^) which is absolutely continuous with 
respect to R . P R, and with its Radon-Nikodym derivative of the special form: 

P = /o(Xo) exp Vt{Xt) d?j g,{X,) R (1) 

where fo,gi : X [0, oo) are nonegative Borel measurable functions on X, the potential 
: [0, 1] X A" — )• M is also assumed to be Borel measurable on [0, 1] x A" and all of them 

satisfy integrability conditions such that ([1]) defines a probability measure. We also say 

for short that P is an h-process. 

It is easy to show (Proposition 14.21 below) that P inherits the Markov property from 

R. Consequently, it is tempting to know more about its infinitesimal generator. The aim 

of this article is to derive the generator of the Markov process P without assuming too 

many regularity conditions on R, fo,gi and V. 

Usual /i-transform. Motivated by potential theory, the special case when V = but 
the terminal time t = 1 is replaced by some stopping time r : 

P = fo{Xo)h{X,) 

has been introduced in 1957 by J.L. Doob |Doo57l IDooOOj with R^'^^ a Wiener process R 
killed at the exit time r of a bounded domain D of Mf^. In this situation, for alH > and 
X in D, the transition probability distributions of P are given by 

P{Xt edz\Xo = x) oc ht{z) R^t\dz \ Xq = x) 

where oc means "proportional to" and z ht{z) = Enll^T-^tyh^Xr) \ Xt = z] is a space- 
time harmonic function on D; this explains the letter h. 

An example. In this paper, we shall only be concerned with the transform defined by 
([1]), without stopping times. As an example, suppose that the reference process R is the 
unique solution of some stochastic differential equation 

dXt = h{Xt) dt + a{Xt) dWt 

with locally Lipschitz coefficients h and o", where is a standard Wiener process on 
X = R"^. This implies that i? is a solution of the martingale problem MP (6, a) : 

R G MP(6,a), 

with b an adapted (drift) vector field and a = aa* an adapted (diffusion) matrix field. 
Since P <^ R, Girsanov's theory tells us that there exists some adapted vector field /3 
such that P solves 

P G MP{b + af3,a). 

Now the problem is to express /3 in terms of the ingredients a, b, /o, gi and V. Specifying 
the abstract results of this article to this continuous diffusion case leads to the next result 
(see Theorem 15.41 below) : The additional drift term /3 can be written as 



(3{t, x) = V^ip{t, x), dtPtidx)-a.e. 



(2) 
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where 



ip{t,x) := log Eh 



exp 



Vs{X,)ds)giiX^) \Xt 



X 



dtPt{dx)-a..e. (3) 



is defined dtPt{dx)-a.e. and V'^ is some linear operator which we call the P- extended 
gradient. This gradient coincides with the usual one on smooth functions: V^u = Vu, 
for all u G C^(ffi'^), when the diffusion matrix a has full rank. Of course, if R admits a 
regularizing and positivity improving transition probability density (for instance if R is 
the Wiener measure) and V = 0, then ipit^x) := \ogEji{gi{Xi) \ Xt = x) is well-defined 
and smooth on [0, 1) x and /3 = Vip. This situation is investigated in details by H. 
Follmer |Fol88j . On the other hand, when V" is a non-regular measurable function, even 
if R admits a regularizing semigroup, may be a non-regular continuous function and 
([2]) has an unusual meaning. 

Non-regularity of V . The transition probability distributions in both directions of time 
of the generalized /z-transform P are the Euclidean analogues [CZ911 ICZOSj of the Feyn- 
man propagators [FH65j in the sense that for all t G [0, 1] 



P{Xt edz\XQ = x) oc E_ 



R 



exp 



V,{Xs)ds]g^{X{)\Xt 



P{Xt edz\Xi = y) oc E_ 



R 



/o(Xo)exp 



Vs{X,)ds I X, 



R{Xt edz\XQ 



R{Xt edz\Xi 



As non-regular potentials V are usual in physics, for instance discontinuous potentials 
with vertical asymptotic directions, we do not even assume that V is continuous. 

From another view point, ([T]) is the generic form of the solution of the minimizer of the 
relative entropy 



H{P\K) ■- / log 



^)dPe|0,oo| 

which is seen as a function of P, subject to the constraints that its initial law Pq is equal to 
some given /xq G P(A') and its flow of time-marginal laws {Pt)ti^[o,i] solves some prescribed 
Fokker-Planck evolution equation. In this convex optimization problem, fo^gi and V 
act like Lagrange multipliers. See |Csi75l IFolSSl ICL941 ICL951 ICL96] for related entropy 
minimization problems and |Leo01] for a convex analytic derivation of this statement. For 
instance, when motivated by stochastic mechanics |Nel88] . the above mentioned Fokker- 
Planck equation is related to the solution of some Schrodinger equation and its drift term 
explodes on the (nodal) set where the wave function vanishes. This enforces irregularities 
of V. See the introduction of |MZ85j for a brief explanation of this point and also Eq. (8) 
of |MZ85j where the potential Vt{x) = ^^(x) appears, with the Markov generator 
of R and $ the wave function. 



Previous approaches to this problem. Let P G P(^^) be a Markov process and 
Tf^u{x) := Ep[u{Xt) \ Xs = x], u & U, < s < t, he its semigroup on some Banach 
function space {U, \\ -Wu)- For instance U may be the space of all bounded Borel measurable 
functions on X equipped with the topology of uniform convergence. Its infinitesimal 



generator is A = {Ai 



he[o,i 



with 



A^uix) 



u{Xt) I Xt 



u G domA 



(4) 
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where the domain domA^ of is precisely the set of all functions u eU such that the 
above strong limit exists for all t G [0, 1) and x E X . We have seen with ([2]) and (|3]) that 
the function g defined by 



9t{x) := E_ 



R 



exp (y- j^^ V,{X,)d^ gi{X,)\Xt 



X , dtPt{dx)-a..e. (5) 

plays an important role in the description of the dynamics of P. One can prove rather 
easily (see |RY99j for instance) that when g is positive and regular enough, the generator 
A^ of the Markov semigroup associated with P is given for regular enough functions u 
on X, by 

Afu{x) = A%{x) + ^^^^^(t, x), (t, x) G [0, l]xX (6) 

9t 

where F is the carre du champ operator, defined for all functions u, v such that u, v and 
the product uv belong to the domain domA^ of A^, by 

r{u, v) = A^{uv) - uA% - vA%. 

For Eq. to be meaningful, it is necessary that for all t e [0, 1], gt and the product gtu 
belong to domA^. But we have already noticed that with a non-regular potential V, g 
might be non-regular as well. There is no reason why gt and gtU are in dom A^ in general. 

Clearly, one must drop the semigroup approach and work with semimartingales or 
Dirichlet forms. The Dirichlet form theory is natural for constructing irregular processes 
and has been employed in similar contexts, see |Alb03] . But it is made-to-measure for 
reversible processes and not very efficient when going beyond reversibility. Let us have a 
look at the semimartingale approach. Working with semimartingales means that instead 
of the infinitesimal semigroup generators A^ and A^, we consider extended generators in 
the sense of the Strasbourg school |DM87j . see Definition l2.2l below. This natural idea has 
already been implemented by P.-A. Meyer and W.A. Zheng |MZ84t IMZ85] in the context 
of stochastic mechanics and also by P. Cattiaux and the author in |CL94l[CL96j for solving 
related entropy minimization problems. But one still had to face the remaining problem 
of giving some sense to r{gt,u). Consequently, restrictive assumptions were imposed: 
reversibility in [MZ85J and, in |CL96j . the standard hypothesis that the domain of the 
extended generator of R contains a "large" subalgebra. In practice this last requirement 
is not easy to verify, except for standard regular processes. In particular, it is difficult to 
find criteria for this property to be inherited by P when P <^ R. 

In the present article, we overcome these limitations by choosing a different strategy which 
is based on stochastic derivatives and in some sense is more direct. 

Further developments. Generalized /i-processes are not only designed for Euclidean 
quantum mechanics ^CZ08j or stochastic mechanics jNel88j . 

(i) They are a valuable tool for obtaining a new look at Hamilton- Jacobi-Bellman equa- 
tions, by comparing the definition ([T]) with the usual Girsanov exponential Radon- 
Nikodym density. 

(ii) Because of the time symmetry of their definition when R is assumed to be reversible, 
they may bring interesting information about time reversal. 

(iii) Even when V is zero, ([1]) provides an interesting process P which is sometimes called 
a Schrodinger bridge. It minimizes H{P\R) subject to the marginal constraints Pq = 
Ho and Pi = /ii. A connection with optimal transport is described in |Leoaj . The fiow 
{Pt)t€[o,i] of this bridge is similar to the displacement interpolation introduced by R. 
McCann [McC95j which is used for deriving functional inequalities or as a heuristic 
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guideline in the so-called Otto calculus, see |Vil09j . This suggests that using {Pt)te[o,i] 
instead of the displacement interpolation could yield interesting results. 

These potential developments will be investigated in future works. 
Outline of the paper. The stochastic derivative of P : 

Lfu{x) := lim —Ep \u{Xt+h) — u{Xt) I Xt = x] , m G domL"^ 
/i4.o h 

(compare (jl])) was introduced by E. Nelson in |Nel67] . As usual, domL-'' is defined to 
be the set of all functions u such that the above limit exists, for the exact definition see 
Definition EH 

As a first step, we show that for a Markov process P, the stochastic derivative is equal 
to the extended generator on a large class of functions u on X : 

C^u{x) = L^u{x), dtPt{dx)-a.e. 

This identity is the purpose of next Section [2] whose main results are Theorem 12.91 and 
Proposition 12.101 The key of Theorem I2.9[ s proof is the convolution Lemma [2. 7[ 

With this general tool at hand, it remains to compute L^u for sufficiently many func- 
tions u to determine the martingale problem associated with P. And in view of with 
gt defined at this essentially amounts to : 

(i) Prove that gt G domL^ and compute L^gf, 

(ii) Prove that gtu G domL^ for many "regular" functions u. 

Problem (i) is solved at Section [3] by means of standard integration technics. 
Problem (ii) is trickier. We solve it at Section H] by assuming that the relative entropy 
of P with respect to R is finite: 

H{P\R) < oo. 

The main technical step for solving this problem is Lemma which allows us not to rely 
on Girsanov's theory in its usual form. In particular our abstract results are valid without 
assuming that R has the representation property (any i?-martingale can be represented 
as some stochastic integral). 

The main result of this paper is Theorem 14. 12[ It extends ([6]). 

At Sections O and Owe examplify our abstract results by means of continuous diffusion 
processes on M'^ and time-continuous Markov chains. The main results of these sections 
are Theorem 15.41 which states ([2]) and Theorem 16.11 which describes the dynamics of the 
/i-transforms of Metropolis algorithms on a discrete countable state space X. 

2. Stochastic derivatives 

We denote for any t G [0,1], Xt := {t,Xt) G [0,1] x X and for any stopping time 
Y; := Yt;,^ and := (t A r, Xt^,). 

Let P be a probability measure on Q. Recall that a process M is called a local P- 
martingale if there exists a sequence {Tk)k>i of [0, 1] U {oo}-valued stopping times such 
that limfc_>ooTfc = oo, P-a.s. and for each A; > 1, the stopped process M^^^ is a uniformly 
integrable P-martingale. A process Y is called a special P-semimartingale i{Y = B + M, 
P-a.s. where P is a predictable bounded variation process and M is a local P-martingale. 
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Definition 2.1 (Nice semimartingale). A process Y is called a nic^ P -semimartingale 
if Y = B + M is a special P -semimartingale and the bounded variation process B has 
absolutely continuous sample paths P-a.s. 

Definition 2.2 (Extended generator of a Markov process). Let P be a Markov process. A 
measurable function u on [0, l\x X is said to be in the domain of the extended generator 
of P if there exists a measurable function v on [0, 1] x A" such that Jj^ ^ \v{t, Xt) \ dt < oo, 
P-a.e. and the process 

■=u{t,Xt) -u{0,Xo) - I v{s,Xs)ds, < t < 1, 

is a local P-martingale. We denote 

v{t, x) =: C^u{t, x) 

and call the extended generator of P. The domain of the extended generator of P is 
denoted by dom£^. 

Remarks 2.3. 

(a) In other words, the measurable function u on [0, 1] x is in dom£'^ if the process 
u{t,Xt) is a nice P- semimartingale. 

(b) The adapted process t i— )■ J^^j v{s,Xs) ds is predictable since it is continuous. 

(c) M" admits a cadlag P-version as a local P-martingale (we always choose this regular 
version) . 

(d) In many situations it is enough to consider continuous functions u. But it will be 
useful at some point to consider C^g with g given by (E]) and it is not clear a priori 
that g is continuous in the general case, see Theorem 14.121 and Lemma 15.31 below for 
instance. This is the reason why we do not restrict dom£'^ to continuous functions. 

(e) The notation v = Cu almost rightly suggests that f is a function of u. Indeed, when 
u is in dom£^, the Doob-Meyer decomposition of the special semimartingale u{t, Xt) 
into its predictable bounded variation part J Vg ds and its local martingale part is 
unique. But one can modify v = C^u on a small (zero-potential) set without breaking 
the martingale property. As a consequence, u i— )■ C^u is a multivalued operator and 
u I— 7- C^u is an almost linear operation. 

(f) Suppose that to is a fixed time of discontinuity of P, i.e. P{Xt^ ^ -^t") > 0- Then, 
in general a continuous function u cannot be in dom^'^. For this reason, one should 
think of the notion of extended generator for processes P that do not have any fixed 
time of discontinuity: P{Xt ^ X^-) = 0, for all t G [0, 1]. 

The notion of generator is tightly connected with that of martingale problem. 

Definition 2.4 (Martingale problem). LetC be a class of measurable real functions u on 
[0, 1] X A* and for each u &C, let Lu : [0, 1] x A* — )■ M &e a measurable function such that 
Jjp y \C,u{t, uj^ \ dt < oo for all uj E Q. Take also a probability measure fio G P(^)- One says 
that Q G P(^^) is a solution to the martingale problem MP{C,C; fio) if Qo = /Uq G 
and for all u ^ C, the process 

u{t,Xt) -u{0,Xo) - Cu{s,Xs)ds 

is a local Q-martingale. 

-'^This is a "local" definition in the sense that this notion probably appears somewhere else with another 
name. 
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As in Definition I2.2[ this local martingale admits a cadlag Q- version. 
Playing with the definitions, it is clear that any Markov law Q G P(f2) is a solution to 
MP{C^ ,C; Qo) where is the extended generator of Q and C is any nonempty subset 
of dom C^. 

Our aim is to show that the extended generator can be computed by means of a sto- 
chastic derivative. 

Definition 2.5 (Integration time). Let u be a measurable real function on [0, 1] x and 

T be a stopping time. We say that t is a P -integration time of u if the family of random 
variables {u{X^)\t G [0, 1]} is uniformly P-integrable. 

Definition 2.6 (Stochastic derivative of a Markov process). Let P be a Markov process 
and u be a measurable real function on [0,1] x X. We say that u admits a stochastic 
derivative under P at time t G [0, 1] if for Pt-almost all x & X there exists a P -integration 
time of u such that > t, P-a.e. and for any P -integration time t of u satisfying 
T > cr^, P-a.e. the following limit 

L^u{t, x) := lim Ep ( — [u{Xl^f^) — u{t, x)] \ Xt = Xj 

exists and does not depend on t. 

If u admits a stochastic derivative for dtPt{dx)- almost all {t,x), we say that u belongs to 
the domain dom of the stochastic derivative of the Markov process P. 
If the function u does not depend on the time variable t, we denote 

L^u{x) = L^u{t,x). 

This extension of Nelson's definition by means of integration times seems to be new. 
It is consistent since the supremum of two integration times is still an integration time. 

Indeed, the supremum of two stopping times is a stopping time and for all t, \u{X^ )| < 

Kx:)i + Kx:')i. 

As in Definition [221 we do not restrict the domain of the stochastic derivative to continuus 

functions, see Remark [2. 3l -(d). 

Since P is a Markov process, we have also 

L^u{t, x) = lim Ep ( — [u^X^j^f^) — u(t, x)] \ t > t, Xt = xj . 

We denote P the product of the Lebesgue measure on [0, 1] by the process P : P{dtduj) = 
dtP{duj). In the sequel, we shall be concerned with the function space -L^([0, 1] x VL,P). 

Lemma 2.7. For all h > 0, let > be a measurable convolution kernel such that 
supp C [—h, h] and k^{s) ds = 1. 

Let P be a bounded positive measure on Q (which may not be a probability measure) and 
v{t, u) be a function in L^([0, 1] x Q, P) with 1 < p < oo. Define for all h > and t G [0, 1], 
k^ * v{t) = Jjg k'^it — s)vs ds where v is extended by putting Vs = for all s [0, 1]. 
Then, k^ * v is m Lp{[0, 1] x fi,P) and liuihiok^ *v = v m Lp([0, 1] x fi,P). 

We see that k^{s)ds is a probability measure on M which converges narrowly to the 
Dirac measure 5q as h tends down to zero. 

Proof. In this lemma, we endow as usual Vl with the Skorokhod topology which turns it 
into a Polish space and has the interesting property that its Borel cx-field matches with 
the cylindrical cr-field. 
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We denote L^([0, 1] x n, P) = L/iyP) and start the proof by showing that k'^ * v E 
LP(P). For P-almost all v{-,uj) G Lp{[0, 1]) so that * v{-,uj) is also in Lp([0, 1]) with 
\\k'^ * v{-,u)\\lp(^[o^i]-) < ||t>(-, a;)||2,p([o,i]). It remains to integrate with respect to P{du) to 
obtain 

* ^IIlp(p) < II^IIlp(p) < oo- (7) 

Now, we prove the convergence. As p is finite, the space Cc([0, 1] x Q) of all continuous 
functions with a compact support in [0, 1] x f2 is dense in U'{P). We approximate v in 
U^P) by a sequence {vn)n>i in C'c([0, 1] x Vt). For all h and n 



i_LP(P) ^ II A; * (f — f.„ 
< II A;'^ * ti„ — V. 



Ilp(P) + ||A; * ^ ^nllLP(P) + 11^" ~ ^IIlp(P) 



"IILp{P) ^nllLP(P) 

where we used ([7]). 

Take an arbitrary small > and choose n large enough for ||f — Vn\\]^v(p) < ?7 to hold. 
Then, 

\\k^ *v - w||ip(p) < II A;^ * Vn - II LP{P) + 2?7- (8) 

Fix this n. Since Vn is in Cc([0, 1] x it is a uniformly continuous function. Therefore, for 
all > 0, there exists hirf) > such that for any t, t', oj, oj' satisfying \t — t'\ + dQ^u, u') < 
h{ri), we have \vn(t',u') — Vn{t,ijj)\ < r], where dfi is the Skorokhod metric on Q. In 
particular, with u = u', we see that 

\t' — t\ < hirf) =^ sup \Vn(t' , Uj) — Vn(t, IjJ)\ < T]. 

Because of the property: supp A;^ C [—h,h], we deduce from this that for any u E fl, 
\k^*Vn(t) — f„(t)| < /jg \vn(t — s) —Vn(t)\k^{s) ds < 7] a.s soou as h < h{ri)/2. Consequently 
\\k'^*Vn—Vn\\LP(p) ^ P{^)r]- Finally, with (jH]) this leads us to || A;'**?; — f ||^p(p) < {2+P{^l))ri. 
Since rj is arbitrary, this shows that lim/i^o ||A;'^ * f — v\\j^p(p-j = 0, which is the desired 
result □ 



Proposition 2.8. Let P be a Markov process and u be a function in the domain dom£^ 
of the extended generator of P. We suppose in addition that there exists 1 < p < oo 
such that Ep Jj^ ^ \C^u{t, Xt)\P dt < oo. Then, 



lim Ep 



[0,l~h] 



-Ep[u{t + h,Xt+h) 



u{t,Xt)\Xt]-C''uit,Xt 



dt = 0. 



(9) 



Proof. We denote Vt = C^u(t, Xt). Choosing the specific convolution kernel k^ = j^l[_h,o]y 
and relying on the very definition of the extended generator, we obtain 

1 
h 



^Ep[u{t + h,Xt+h)~u{t,Xt) I Xt] = ^Ep[u{t + h,Xt+h)-u{t,Xt) I X[o,t]] 



Ep[k^ * v{t) I X[o,t]] = Ep[k'' * v{t) I Xt]. 



On the other hand, by Jensen's inequality and Fubini's theorem 



Er 



[0,1] 



Epik'^^vit) \Xt]-Vt 



dt 



< E 



Ep [ Ep[k''*v{t)-Vt\Xt 

J [0,1] 

\k^ * v{t) - vt\P dt. 



dt 



[0,1] 
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Our hypothesis v G -^^^([0, 1] x P) is precisely the assumption of previous Lemma [2.71 
which insures that lim^io-^'P /[q i] * '^(^) ~ Vt\^ dt = 0. Gathering these considerations, 



we obtain ®. 



□ 



A variant of this proposition already appears in |Fol86] . But it seems to the author 
that its proof is incomplete and that it is difficult to avoid a convolution argument such 
as Lemma [2.71 

Theorem 2.9. Let P be a Markov process and u be a function in the domain dom^-'^ of 
the extended generator of P. Then, u belongs to domL-^ and 

CFu = L^u, dtPt{dx)-a.e. 

Proof. By the definition of the extended generator, there exists a localizing sequence 
{'^k)k>i of stopping times, i.e. such that lim^^oo Tfc = oo, P-a.e. and for all k > 1, the 
stopped process M'^'= where 



Mt = u{t,Xt 



C u(s,X^)ds, 



is a uniformly integrable martingale. By considering the sequence of stopping times 
inf{t G [0, 1]; Jjq^j \C^u{s, Xs) \ ds > k} G [0, 1] U {oo} indexed by > 1, it is easy to show 
that {Tk)k>i can also be chosen such that for each k, is also an integration time of u. 

Let us consider a fixed integration time r of u such that M'^ is a uniformly integrable 
martingale. Denoting v'^(t) = l{t<T-}£-^-u(t, X^) and choosing k^ = j^li_h,o] as in the proof 
of Proposition 12. 8[ we see that |;[m(X[+/i) — u(x].)] —k^* '^'^{t) is a martingale. It follows 
that 



^-Ep[u{Xl^^) 



u{t, XI) I X[o,i]] = Eplk"^ * v^{t) I X[o,i]] = l{t<r}Ep[k'' * v{t) I 



Remark for future use that this implies that 
1 



h 



Ep[uiX,^,) - u{t,Xl) I X[o,]] = l{i<.}-i?p[n(X,+,) - u{t,Xl) \ XJ 



(10) 



Then, as for (Q with p = 1, we obtain 



limE'p 

'^^^ J[0,TA{l-h)] 

and with Fatou's lemma 



-Ep[u{X;^,,) - u{t,Xl) I X[o,]] - C''u{t,Xt: 



dt = 



Ef 



liminf l{t<T} 

[0,1-h] '^-^O 



-Ep[u{Xl,) 



u{t,Xl) |X[o,t]]-£^M(t,Xi 



dt = 0. 



But, since u is in dom£^, lim/ij^o |:-^pN(^+/i) ~ "^{^i^t) I ^[o,t]] appears as the compu- 
tation of the derivative of an absolutely continuous function. Therefore, this limit exists 
for Lebesgue-almost all t and the lim inih\.Q arising from the application of Fatou's lemma 
is a genuine limitH. Hence, 



E, 



l{t<T} lim 

[0,l-/i] 



h 



Ep[u{X 



t+h) 



^i(t,Xi) |X[o,t]]-/:^M(t,Xi 



dt = {] 



^The absolute continuity plays a crucial role. Note that it is also of primary importance in the definition 
of the extended generator. 
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and with ( ITOj) this shows us that for P-almost all (t, u) we have 

l{rH>*} 1™ ^^i^ [^i^l+h) -u{t,Xt{u)) I X[] (w) = l{^(^)>f}£^M(t,Xt(u;)). 

As the left-hand side vanishes when t{uj) = t, we obtain 

lim [u{X+h)-<t,Xt) \ T>t,Xt]= l{r>t}C''u{t,Xt). 

This results holds true for any integration time r of u such that M'^ is a uniformly 
integrable martingale. 

By assumption, for P-almost all {t, uj) there exists u) large enough for the localizing 
time Tk(t,ui) to satisfy Tk{t,uj){ijj) > t. Choosing cr'^'^'^) = Tk[t,u}), we obtain for (itPt((ix)-almost 
all {t, x) an integration time > t such that any integration time r > satisfies 

lim— _Ep [u{X]_^_fj) — u{t,x) \ Xt = x] {u) = C^u{t,x). 

This completes the proof of the theorem. □ 
Let us investigate a partial converse of Theorem | 



Proposition 2.10. Let P be a Markov process, u and v he measurable real functions on 
[0,1] X X which satisfy the following requirements. The function v verifies J^^ ^ |f(t,Xt)| dt < 

oo, P-a.s. and there exists a sequence {T^)k>i of integration times ofu such that lim^^oo Tfc = 
oo, P-a.s. and for each k > 1, 



lim£^P 



dt = (11) 



Then, u belongs to dom£^ and domL'^ and 

C^u = L^u = v, dtPt{dx)-a.e. 

Note that if P admits a fixed time of discontinuity, there might be many continuous 
functions u which do not verify ffTTl) . 

Proof. The proof relies on the subsequent easy analytic result. 

Claim. Let a,b be two measurable functions on [0, 1] such that a is right continuous, b is 
Lebesgue-integrable and lim/i^o /[g i^h] IH'^(^ + h) — a{t)} — b{t)\ dt = 0. Then, a is abso- 
lutely continuous and its distributional derivative is a = h. 

To see this, remark first that t (-)■ l{o<t<i-h}x{^(^ + h) — a{t)} is integrable for any 
< /i < 1. Take any < r < s < 1. On one hand, we have lim/i|o /[^^j + h) — 

a{t)} dt = b{t) dt and on the other one: /j^^^j l{a{t + h) - a{t)} dt = \ /[^,,_^_^] a{t) dt - 
J /[s s^fi] O'it) dt, so that with the assumed right continuity of a we have lim/i|o /[^^j j^Wii^ 
h) — a{t)} dt = a{s) — a{r). Therefore a{s) — a(r) = J^^^^b{t)dt which is the claimed 
property. 

Let us fix t'' as in the assumption of the proposition. We write E = Ep, ut = u{X^'') 
and Vt = l{t<Tk}v{X^^) to simplify the notation. Define the family of stopping times 
(Tfc := inf{s G [0, 1]; J^q^j \v{t,Xt) \ dt > k} where k describes the integers. By considering 
the stopping times cxfe A r^, we can assume without loss of generality that 

V G L\P). 
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Fix < r < 1. We have 
1 



11 



E 


l[r,l-h] ( 







h 



{ut+h - ut} -vt] dt\X, 



< E 



.J[r,l-h] \ 



+h - Ut} - vt\ \ Xt] dt\ X, 



With f lTTj) and Fatou's lemma, we obtain 

1 



( hminf 


E 


l[r,l-h] ( 


Y hlO 







h 



{ut+h - Ut} - Vt] dt\ X, 



< 



limE [ E(\]-{ut+h-ut}-vt\\Xt) dt = 0. 



Hence, there exists a sequence {hn)n>i of positive numbers such that hm„_j,oo hn = and 



hm 

n— >oo 



E 



[r,l-h„] 



-Ut}-Vt] I Xr 



dt = 0, P-a.e. 



It remains to apply the result of the above claim to a{t) = E[ut \ Xr] and b{t) = E[vt \ Xr 



to see that for all < r < s < 1, -E 
M"^*^ is a P-martingale where 



Us-Ur- /j.^. Vt dt I X[o . 

M,:=u{s,X,)-u{0,Xo)- [ v{t,Xt) 

J\0,s] 



0. This proves that 



dt. 



With the assumptions that limfc_j,oo Tfc = oo, P-a.s., l[o,Tk]U G L^{P) the fact that 
{u(Xl'');t G [0,1]} is uniformly P-integrable by the very definition of the integration 
time Tfc, we conclude that M is a local P-martingale. Therefore, u belongs to dom£^ and 
C^u = V. And we also have u G domL^ and L^u = C^u by Theorem 12.91 □ 



3. FEYNMAN-KAC PROCESSES 



Let P be a probability measure on Q which is a stationary Markov process with the 
invariant probability measure 



m:= RteF{X), VtG[0,l]. 



(12) 



We also consider a lower bounded potential V, i.e. a measurable function V : [0,1] x X ^ 
M such that 

inf V > -Xo (13) 

with < Xo < oo. Let gi be a nonnegative m-integrable function on X . In this section we 
look at the real valued process 



Gt ■.= E 



R 



exp K(X,) ds^ (Xi) I X[o,i] 



--■.gtiXt), te[0,l], g,>0 (14) 



which we call a Feynman-Kac process. Last equality, where gt : X [0, oo) is a measur- 
able function, is a consequence of the Markov property of P. 
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Orlicz spaces. The mere integrabihty of gi is sufficient for defining G, but it will not 
be enough in general for our purpose. We are going to assume that gi is in some Orlicz 
space 

L'^{m) :— : A" — )■ R; measurable, j '^{ao\u\) dm < oo, for some Oo > oj- 

associated with the Young function 7. Recall that 7 : IR — )> [0, 00] is a Young function if 
it is convex, even, lower semicontinuous and 7(0) = 0. Important instances are 

- 7(a) = 7p(a) := with 1 < p < 00, then U'p{m) — U'{m); 

- 1(a) = ,M := { ^ ■ '"^ = 

Let us introduce the functions 

e{a) := e^-a-l, a G M, (15) 
6*{a) := (a + 1) log(a + 1) — a, ae[— l,oo) 

with the convention OlogO = 0. They are convex conjugate to each other and 6{a) = 
logEe"(^-^) where N is a Poisson(l) random variable. Moreover, 6{\a\) and 6'*(|6|) are 
Young functions which are also convex conjugate to each other. 
Two other important Orlicz spaces are 

- ^{a) — 9{\a\) corresponds to the following L'^ : 

L'^''^{m) := jii : A" ^ M; measurable, J e""'"' dm < 00, for some Oo > o| , 

- 7(a) = 6'*(|a|) corresponds to the following L"/ : 

LlogL(m) := i^u : A" — >■ R; measurable, J \u\ log,,. \u\ dm < 00 j- , 

where we use the assumed boundedness of the positive measure m in the above expressions. 
The Luxemburg norm of L^im) is defined by ||M||L7(m) := vai{ot > 0; j;,^l{\u\la) dm < 1}. 
Let 7*(6) := sup„>o{a6 — 7(a)} G [0, 00], 6 > 0, be the convex conjugate of 7. It follows 
immediately from Fenchel's inequality ab < 7(a) + 7* (6), that the Holder inequahty 



\uv 



iLi(m) < '^\VA\L'i{m)\\v\\L'i*{rri): ^ G (m) , V G L^* (m) 



holds true. In particular, since ^^(| ■ |) and ^^*(| ■ |) are convex conjugate to each other, we 
have ||Li(m) < 2||M||LiogL(m)||^^IU«='p(m), for aU u G LlogL{m),v G L'=''P(m). 
The Young function 7 is said to satisfy the condition A2 if there exist constants C,A>0 
such that 7(2a) < Cj{a), for all a> A. The spaces L log L(m) and U'{m) with 1 < p < 00 
satisfy A2. But L^{m) and U^^^{m) do not. 

Preliminciry results. We assume that the next finite entropy condition is satisfied 

i/i > 0, / gi log^ gi dm < 00 
Jx 

and we pick a Young function 7 such that 

/ l{gi) dm < 00 and L log L(m) C L'^{m) C U'{m) for some 1 < p < 00. (16) 
Jx 
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In particular, we have 7 G A2. 

Because of (II2D, (USD and ([IS]), with Gt given by ([H]), we have for all t G [0, 1] and a > 0, 

/ ^{gt/a) dm = ER-f{Gt/a) < ER-i{e'"'Gi/ a) < C^,xER{Gi/a) 
Jx 

where C^,Ao > is some finite constant which can be derived by means of the condition 
A2. Optimizing in a leads us to 

IIS'tlU^M < C'^.aJIs'iIUtM, Vt G [0,1]. 

Recall that a real valued process G is said to admit a cadlag version if there exists a 
modification G' of G, i.e. R{Gt 7^ = for all t G [0, 1], with its sample paths in 
Dm :=/^([0,1],M). 



Lemma 3.1. Let us assume that in addition to ( fT2|) . (|T3|) and (|T6|l . we have 



[0,1] 



V^llLi(m) (it < 00. 



(17) 



Then, the process G admits a cadlag version. In the sequel G will always he assumed to 
he this D^-valued version. 

It is a nonnegative semimartingale which satisfies the so-called Feynman-Kac semigroup 
property: 



E 



R 



exp(^-^ Vr{Xr)dr^Gt\Xio,. 



Gs, 0<s<t<l. 



Moreover, denoting G^ := sup^g^,!] C^t, we have 

\\G*\\l-i{R) < G^.Ao llfi'lllLT(m) 

for some finite positive constant C^,Ao • 

This implies that {'-f{Gt)',t G [0, 1]} is uniformly integrahle in L^{R). 



Proof. Let us prove ([T8|) . For all < s < t < 1, 



Er 

Er 
Er 
Er 



exp (^-^ Vr{Xr) dr^ Gt\X[o,. 
exp ^ ^ VriXr) dr^ Er jexp j^^ dr^ G^ \ X[o,t] | 

Er jexp j Vr{Xr) dr^ Gi I X[o,t]| I X[o„ 



Xu 



exp 



.1] 



VriXr) dr Gi I X| 



which is (|T8l).^ 

Let us define := 1/ + Ao > and for all t G [0, 1] 



Gt := e-^-'-^-'^Gt = E_ 



R 



exp 
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where Gi = Gi = gi{Xi). Because V > 0, we see that for all < s < t < 1 
EniGtl -^[o,s] ) = Eji 



> E 



R 



exp j ^ Vr{Xr) dr^ Gi \ Xp,^] 
exp j Vr{Xr) dr^ Gi \ X[o,s] 



G. 



In other words, G is a nonnegative submartingale. 

It follows from the fact that the forward filtration satisfies the standard assumptions and 
from a well-known result of the general theory of stochastic processes that G admits a 
cadlag modification (still denoted by G) if t G [0, 1] h-> EjiGt G [0, oo) is a right continuous 
real function. But this latter property is a direct consequence of Lebesgue's dominated 
convergence theorem and the pathwise right continuity of 



t G [0, 1] ^ exp Vr{Xr) dr^ G (0, 1] 



which is satisfied under the assumption f[T7|) : Er Jj^ |Vf(Xf)| dt < oo, which implies that 
/jQ^^j \ Vt{Xt)\ dt < oo, R-a.sE 

Furthermore, we have ER^yiGt) < Erj{Gi) < oo by Jensen's inequality and the sub- 
martingale property. Doob's maximal inequality, which holds for any nonnegative sub- 
martingale and any Young function 7 which verifies ( IT6|l R. tells us that there exists a 
positive finite constant c-y < 00 such that 

II sup 7(Gt)|Ui(ij) < Cy sup ||7(Gt)||Li(ij) = Cj\\'y{Gi)\\Li(R) = c^||7(5'i) IIlim < 00. 
iG[0,l] te[o,i] 

Hence {jlGt); t G |0, 1]} is uniformly integrable in L^{R). Since the product of two semi- 
martingales is still a semimartingale, we deduce that Gt = e^°^^^^'^Gt is a cadlag semi- 
martingale such that {7(G();t G [0, 1]} is uniformly integrable in L^{R). This completes 
the proof of the lemma. □ 

Recall that since i? is a bounded nonnegative measure, a family {Hf, t G [0, 1]} of real 
valued measurable functions is uniformly integrable in L}{R) if and only if there exists 
an increasing convex function ^ : [0, 00) — )■ [0, 00) such that Imia^oo ^{o) / a = +00 and 
suPte[o,i] ER^{\Ht\) < 00. 

Claim. Let At,Bt, t G [0,1] be two random variables such that both {'~f{At);t G [0,1]} 
and {'j*{Bt);t G [0,1]} are uniformly integrable in L^{R). Then, the family of products 
{AtBt]t G [0, 1]} is uniformly integrable in L^{R). 

Let us prove this claim. By hypothesis there exist two functions ^1 and .^2 as above such 
that supj £^(^1(7(^4)) < 00 and sup^ i?^2(7*(-Bt)) < ^ where we wrote sup^ = supjgjo,i] 
and E = Er for short. Let ^ be the convex envelope of x ^i(a;/2) A^2(a^/2). It is convex 
as a definition and still increasing and satisfies limri;^oo^{x)/x = 00. We also obtain 
with Fenchel's inequality ^{\AtBt\) < ^{-f{At) + rW) < ^{2-f{At))/2 + ^{2r{Bt))/2 < 



■^Remark that without the assumption that /jq 1] |^i(^t)|'^^ < 00, i?-a.s. and with the convention 
g-00 = 0, i £ [0, 1] M> exp (— Vr{Xr) dr\ e [0, 1] is weh-defined i?-a.s., but it might fail to be right 



continuous 
4 



For Doob's inequahty in the class LlogL, see |RY99[ p. 54] for instance. 
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^i(7(A)) +6(7*(5t)) for each t G [0, 1]. Consequently, sup^ A^tl) < oo. This shows 
that {AfBt] t G [0, 1]} is uniformly integrable and completes the proof of the claim. 

The assumption ( |T71) will not be strong enough for our purpose. We strengthen it in 
the next lemma. 

Lemma 3.2. Let us assume in addition to f|T2l) . f|T3|) and f|T6l) that the family {7*(Vt);t G 
[0, 1]} is uniformly integrable in L^{m). Then, 

(1) { ^ i+^i] I K I \Gt+h — Gt\',t ^ [0, l],/i>0} is uniformly integrable in L^{R); 

(2) \^^^^GtVt{Xt)dt IS m L\R)- 

(3) {VtGt{Xt)]t G [0, 1]} is uniformly integrable in L^{R). 

Proof. We write Vt = Vt{Xt), sup^ = sup^gjg^i] and E = E^ for short. 

• Proof of (1). There exists a function ^ as above such that sup^E^{'y*{Vt)) < oo. 

But E^ (i Y{Vs) ds^ < i Eai*{Vs)) ds < sup, Ei{Y{Vt)) < oo. This shows 

that {ji J[f^_^_h]1*(ys) ds]t G [0,1], /i > 0} is uniformly integrable. On the other hand, 

we already know by Lemma ISTTl that {'y{\Gt+h — Gt\);t G [0, l],h > 0} is also uniformly 
integrable. The above claim permits us to conclude. 

• Proof of (2). We see that 

E [ Gt\Vt\dt<E{^{G,)) + E I ^*{Vt)dt<E{^{G,)) + sn^E^\Vt)<oo 
J[o,i] J[o,i] t 

which is finite by Lemma 13.1 1 and the assumption that {7*(Vf);t G [0,1]} is uniformly 
integrable. 

• Proof of (3). The result directly follows from the above Claim, Lemma [3.11 and our 
assumptions on V. □ 

The extended Feynman-Kac generator. The main result of this section is the next 
theorem. 

Theorem 3.3. Let us take the following ingredients, 
(i) R & P(^) o stationary Markov process with invariant law m = Rt E P(A') for all 
tG [0,1]; 

(a) •y is a Young function which satisfies (|T6|l and Y is its convex conjugate; 
(Hi) V is a measurable function on [0,1] x X which is bounded below and is such that 

{l*iVt)]t G [0, 1]} is uniformly integrable in L^{m); 
(iv) gi is a nonnegative function on X in L'^{m). 

Then, the function g : (t, x) E [0,1] x X 9t{x) G [0, oo) which is defined for all t G [0, 1], 
m-almost everywhere by 



9t{Xt) ■■= E 



R 



R-a.s., 



exp dsj ^7i(Xi) I X[o,t] 

is a nonnegative function in L'^([0, 1] x X, dtm{dx)) which is in domL^ and in dom£^. 
Moreover, it satisfies 

L^g{t,x) = C^g{t,x) = Vt{x)gt{x), dtm{dx)-a.e. 

and -^^^^^\Vt{x)\gt{x) dtm{dx) < oo. 

Proof. The proof is based on an application of Proposition 12.101 with u{t,x) = gt{x) and 
v{t, x) = Vt{x)gt{x). We write Vt = Vt{Xt) and E = Eji for short. 

We know by Lemma 13.21 that Jj^ \Vt{x)\gt{x) dtm{dx) = E J^^ \ VtGt\ dt < oo. This 
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implies that \ VtGt \ dt < oo, i?-a.s. We have also seen at Lemma [3?T] that Gt is a right 
continuous uniformly integrable process. It follows that we can choose = oo R-a.s. for 
all /c > 1 in formula ([II]) and that for all < s < t < 1, t G [s, 1] i-)- E{Gt \ Xg) is a 
right continuous real function. Therefore, to obtain the announced results, it is sufficient 
to show that 

■ 1 



limii^ 



[0,1-h] 



h 



Ep[G 



t+h 



Gt I Xf] - VtGt 



dt = 0. 



We decompose 



where 



--E[Gt+h - Gt I Xt] + VtGt = Ep[Ai + 5* + | Xt, 



4* 



ci 



V,ds\Gt 



[t,t+h] 



t+h] 



Vsds 



-1 (G 



t+h 



Gt 



Gt^ 



{Vt - Vs) ds 



[t,t+h] 



with e{a) := 



limii^ 



a — 1, a G M which we already met at ( 1151) . It remains to prove that 

\C{\dt = Q. 



[0,l-/i] 



\Ai\ dt = limE 
HO 



\Bi\dt 



limE 



[0,1-h] 



Proof of lim/i^o E /^^ \Al^\dt = 0. We have 



0<-9 
h 



Vsds 



< 



[t,t+h] 
1 
h 



K ds 



[t,t+h] 



Vsds 



I VJ ds. 



[t,t+h] 



But EY {i!yt,t+h] \ys\ds^ < {j^^^^^^^EY{\Vs\)ds < &nVtEY{\Vt\) < oo by assumption. 
Since limf,_>oo 7*(&)/& = cxd because 7 doesn't grow too fast, {A\]t G [0,1], /i > 0} is 
uniformly integrable. This leads us to the desired convergence result since lim/i^o ^K*^) = 
for (iti?((iw)-almost all G [0, 1] x Vt. 

• Proof of \mihioE l-B/il dt = 0. Since t L ^ Vgds is absolutely continuous and 



I [0,1-h] 

t h-^ Gt is right continuous -R-a.s., we see that 

1 



\Bi\<e 



Xoh 



h 



[t,t+h] 



Vs\ ds \Gt+h — Gt\ — 0, R-a.s. 

hiO 



On the other hand we have shown at Lemma [3.21 that {Bj^; t G [0, 1], /i > 0} is uniformly 
integrable. 

• Proof of lim/iio E /^^ dt = 0. We have 



E 



Gt 



[0,l-h] 



(Vt - Vs) ds 



[t,t+h] 



dt<E 


G. [ 


Vt- \ ! Vds 

J[t,t+h] 


dt 




J[0,l-h] 





where we put Vt = for all t > 1. By Lemma 13.1^ G L"'{R). Therefore the mea- 
sure G^:R is a bounded measure and we can apply Lemma [2171 with v(t,u) = Vt{u) in 
L^([0, 1] X Q, G^:R) and = j^l[_h,o]- This completes the proof of the theorem. □ 
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4. Generalized /i-transforms of a Markov process 

Let R G P(^^) be a stationary Markov process with the invariant probabihty measure 
m G as in Section [31 In the present section we consider the process 

P := /o(Xo) exp ^ ^ VtiXt) d?j g^{X^) R G P(fi) (19) 

where : [0, 1] x A* — )■ R is a lower bounded measurable potential and 

/oGL^*(m),(?iGL^(m), /o,^?i>0. (20) 
It is assumed once for all that 

i?(/o(Xo)(7i(Xi) >0) >0 

to discard the uninteresting trivial situation where P = 0. We normalize /o and gi to 
obtain P{Q) = 1. 

Remark that exp ^— Jj^ V^(X() rft j is bounded. It follows with the assumption (1201) 

that /o(Xo) G L^*iR),g,iXi) G L^(i?) and that /o(Xo) exp (- /^^ ^ V;(XO rft) (7i(Xi) 

is a nonnegative i?-integrable function. Hence, it can be normalized such that P is a 
probability measure. 

Definition 4.1 (Generalized /i-transform of R). Let R G P(^^) be a stationary Markov 
process which admits an invariant probability measure. 

A process P G P(^^) which is specified by formula (fT9l) is called a generalized h-transform 
of R, or a generalized h-process for short. 

It is not essential that R is assumed to be a stationary Markov process in this definition. 
Our aim is to identify P as the solution of a martingale problem. To do it, we are going 
to derive the extended generator of the generalized /z.-process P on a class of functions 
C which is large enough to characterize P. With Theorem 12. 9[ we see that we are on the 
way to compute its stochastic derivative on C. 

Playing with the Markov property. Recall that P G P(^^) is a Markov process if and 
only if for all t G [0, 1], X[o^t] and X[t,i] are independent with respect to the conditional law 
P(- I Xt). In other words, if and only if the past and future are independent conditionally 
on the present. This property is invariant with respect to time reversal. In particular the 
time reversed process of R is still Markov. As with the definition of g at f|T^ . one can 
define a measurable function ft{x) on [0, 1] x A* by the formula 



E 



R 



/o(Xo)expf- / Vs{Xs)ds) |X[i,i] 
V J[o,t] J 



--■.ftiXt), tG [0,1] P-a.s. (21) 



since pR(a | X^^^i]) = Efi{a \ Xt) for any X[o,t] -measurable and integrable function a. As 
exp ^— Jjg^j Vs{Xs) ds^ is bounded and /o(Xo) G L'^*{R), we see that ft G L'^' {m) for all 



t G [0, 1] 

Proposition 4.2. 

(1) The generalized h-process P is Markov. 

(2) For every t G [0, 1], P^ ^ m and 

dPt 



dm f^'^ (22) 
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where ft and gt are defined respectively by fl2Tl) and f|T4|) and stand respectively in 
L"* (m) andU'{m). 
(3) For every < s < t < 1, 



dR\^ f] dm 



exp ^ ^ Vr{Xr) dr^ gt{Xt) (23) 

(24) 



fs{Xs) exp j^^ ^ Vr{Xr) dr^ f,(Xty'^{Xt) 
f,iXs)exp(^- VriXr)dr^gtiXt) 



where no division by zero occurs in the sense that gs > 0, Ps-a.s. and ft > 0, 
Pt-a.s. 

Proof. • Proof of (1). Fix < t < 1 and take two bounded nonnegative functions 
a and b such that a is X[o,t] -measurable and b is X[j^i]-measurable. Let us write a = 

/o(Xo) exp /jQ^^j Vs{Xs) ds^ G cT(X[o,i]) and (3 = exp /j^ ^ K(Xs) ds^ gi{Xi) e (T{Xit,i] 
so that P = a/3 R and 

Epiab I X,) = ^^(^^f , ' ^ i^^(ac. | X,)E^(y | X,) ^ ^^^^ | ^^^^^^^ | 



where we used the Markov property of R at the marked equahty. This proves that P is 
Markov. 

• Proof of (2) and (3). As a general result of integration theory, if P = ZR with Z G 
L^{R), then the push-forward P^ := 0#-P of the measure P by the measurable application 
is absolutely continuous with respect to Rip := 4>#R and P<^ = Er{Z \ (p) R^ where 
Er[Z I 0) := Er{Z I cr(0)) is the conditional expectation of Z with respect to the cr-field 
cr(0) generated by 0. In particular, with = Xys,t\ we obtain 



= Eji{dP/dR I X[s,t]) 



We have 



EnidP/dR I X[,,4] 
Er 



E 



/o(Xo) exp K(X,) rfr j (?i(XO | X[,,i] 

/o(Xo) exp f- ( / + / + / 1 K(X,) dr") ^/i(Xi) | X[,,i] 

V L^[0,sl J\t,l]) / 



= /,(X,)exp(^-y^ Vr{Xr) dr j gtiXt) 

where the Markov property of R is used at last equality. In particular, when s = t 
this gives us (1221) . But with (l22l) . we see that for all t, ft > and (7* > 0, Pj-a-s., 
/,(X,) = ^,(X,)-i£(X,) and <7*(X,) = /,(X,)-i£(X,). □ 

A preliminary result under a finite entropy condition. A seemingly innocent result 
is proved at Proposition 14.71 below. But in fact it is a mendatory technical key to our 
approach. It states that, provided that the canonical process is a nice P-semimartingale 
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(see Definition I2.ip . under the assumption that the relative entropy 



f dP 
H(P\R) := log — dP <oo 
J dn 



is finite, if a large class of regular functions stands in dom£^, then it is also in dom£^. 

Let r be a probability on such that the canonical process x on D]^ is a nice semi- 
martingale 

x = xo + B + M\ r-a.s. (25) 

where B is an absolutely continuous process and iVf is a local r-martingale. Suppose also 
that the quadratic variation and the jump compensator are absolutely continuous. More 
precisely, there exists a nonnegative adapted process a such that f^^-^^atdt < oo, r-a.s. 
and 

d[x,x]1 = atdt, r-a.s. 

and the dual predictable projection J of the jump measure ^o<s<t ^(sA^s) has the following 
form 

it{dtdq) = dt£t{dq), r-a.s. 
This means that it = i{t,X[o,t)', ■) is a predictable nonnegative measure on M^, := M \ {0} 



such that 



/(s,X[o,.); Ax,) = / f{s,xio^,y,q)dsis{dq) + Mi 

i^.^i JlO,t]xM., 



0<s<t 



where is a local r-martingale and this decomposition is valid for any measurable 
function / such that Jj^ ^j^j^ \f(t,X[Q^ty, q) \ dtit{dq) < oo, r-a.s. 
It is also assumed that 

e{a\q\) dtit{dq) < oo, Va > r-a.s. (26) 

[0,1] xM, 

where 9 {a) := e" — a — 1, a G M already appeared at (fT5i) . 

Lemma 4.3. Let r be as above and p be a probability on Z)k such that H{p\r) < oo. 
Then, x is also a nice p-semimartingale. 

Remarks 4.4. 

(1) Girsanov's theorem tells us that if x is an r-semimartingale and p ^ r, then x is 
also a p-semimartingale. This lemma tells us that the property of being a nice 
semimartingale is also hereditary under the stronger condition that H{p\r) < oo. 

(2) In case when no jump occurs and the r-semimartingale is built on a Brownian 
filtration, it is well-known that Lemma is still valid with the weaker assumption 
that p <^ r instead of if(p|r) < oo. This follows from Girsanov's theorem and a 
martingale representation theorem. 

(3) The assumption H{p\r) < oo is not very restrictive. Indeed, p ^ r means that 
dp/dr G L^{r)., while if(p|r) < oo means that {dp/dr) log^ {dp/dr) G L^{tc). 

(4) For more details about extensions of this result, see |Leobj . 

Proof. The proof is based on the variational representation 

H{p\y) = sup{EpU — log-Erc"; u measurable : Ej.e^ < oo} (27) 

of the relative entropy which holds true for any probability measure p such that H{p\r) 
is finite, see for instance |Leobt Lemma 3.1] for a proof. 
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Let h belong to the space S of all simple predictable processes: 

k 

ht = hol{o}{t) + /lil(T„T,+i](t) 

1=1 

with k a finite integer, hi G cr(x[o^Ti))) \hi\ < oo and < Ti < ■ ■ ■ < T^+i = 1 an 
increasing sequence of stopping times. Its stochastic integral with respect to iVf is h-M^ = 
Si=i ^i(^T,+iAt ~ ^f,At)' ^ ^ [0' 1] stochastic exponential of h ■ is 

£(h ■ M'')t = exp (h ■ Ml - [ ^a, ds - [ OiKq) dsis{dq) 

Under the assumption fl26l) . the integrals in the exponential are finite r-a.s. so that the 
sequence of stopping times inf{t G [0, 1]; Jj^^j ds + /^^jxr ^(^s?) dsis{dq) > k} tends 
to infinity r-a.s. as k tends to infinity. It follows that £{h-M^) is a positive supermartingale 
and in particular that: Ey.£{h ■ M^)i < 1. Therefore, for any h in S, log ErS{h ■ M{") < 
and with (1271) we obtain 



Ep(/i-M[) < H{p\r) + Ep([ ^atdt+ ! 



e{Kq)dUt{dq) 

[0,1] ^ -'[0,l]xR, 



^[0,l]xDii 

where 

pidtdr]) = dtp{dri) 

and for all t e [0,1],1] e D^, x eR, 



<^{t,r]-x) ■.= at{vy/2+ / e{\qx\)i{t,r]- dq). 

A standard convexity argument (note that ^(|x|) is a convex nonnegative even function) 
proves that the gauge functional 

\h\p := inf < a > 0; / ^{t,r]]ht{r]) / a)p{dtdr]) < 1 [ G [0, cx)) 

I i[0,l]xDji J 

is a seminorm on S. Considering h/\h\.p and —h/\h\.p in the above inequality, it is easy to 
deduce that 

|^p(/i-M[)|<(i7(p|r) + l)|/i|p, V/IG5. 

This means that if iJ(p|r) < oo, h t— )■ E.p{h ■ Ml) is a | ■ |p-continuous linear form on S. 
But, I ■ Ip is the seminorm of an Orlicz space and by assumption fl2B]) . / (^{ah) dp < oo 
for all a > and h & S. This implies that iS is a subspace of the "small" Orlicz space 
^•f(p) := {/ : [0,1] X ^ M, measurabl e, / r/; aft{r])) pjdtdr]) < oo,Va > 0} 
whose dual representation is well-known, see |RR91j : There exists a measurable func- 
tion k on [0, 1] X which stands in the "large" Orlicz space {k : [0, 1] x — )■ 
M, measurable,/ ^*{t,7];aokt{r]))p{dtdri) < oo, for some > 0} =: L**(p) C ^^"'^(p) 
associated with the convex conjugates r/; ■) of r/; ■), such that 

Ep{h-Ml)= [ kt{r])ht{v)p{dtd7]), V/i G 5. (28) 
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Since h is predictable, we also have J khdp = Jj^ Ep{ktht) dt = Ep J^^ Ep{kt \ X[o,t))/it dt 
and taking ht = Ep{kt \ X[o,t)) we see with ( !28l) that 

( / /it dMl - I htbt dt] =0, Mhe S. 

\i[0,l] ^[0,1] / 

It follows that Mf := Mf — Bt with Bt := /^q 6s ds is a local p-martingale and with f l25|) 
we finally obtain that 

x = xo + fi + fi + MP, p-a.s. 
where B + B has absolutely continuous sample paths p-a.s. □ 
Let us go back to R and P given at (fT9|) . 

Definition 4.5 (The class W/j). Let the reference Markov process R be given. We say 

that the measurable function u : [0, 1] x A" — )■ M zs m the class Ur (with respect to R): 
u G Ur, if 

(a) u E dom£^; 

(b) d[ut{Xt),Ut{Xt)Y < dt, R-a.s- _ _ 

(c) the predictable dual projection o/ ^^^jg AMt(Xt)) satisfies i'^{dtdq) = dti^{dq) 
and Jjg ij^^^ ^(c^kl) dti^{dq) < oo for all a > 0, R-a.s. 

In other words, u G Ur if the process u{t,Xt) is a i?-semimartingale and its law r G 
P(Dir) meets the assumptions of Lemma 

Remark 4.6. For the class Ur, we have in mind C^'^([0, 1] x M'^) when R is such that the 
canonical process X is a nice i?-semimartingale with its values in X = W^. Indeed, at 
least in the continuous case when no exponential moments of are required, Ito's formula 
immediately implies that C^'^([0, 1] x W^) C Ur. 

Otherwise, if X is not a nice i?-semimartingale, then it might happen that Ur reduces to 
the constant functions. 

A useful result is the following 

Proposition 4.7. Let us assume that H{P\R) < oo. Then any u G Ur is also in dom£-^. 

Proof. Let \E' : — )■ be the application ^ = {ut{Xt))teio,i]- The measure r = '^#R G 
P(Dir) is the law of the process {ut{Xt))ti=[o^i] when the canonical process is governed by 
R G P(fi). By the definition of the class Ur, r satisfies the assumptions of Lemma [4.3[ 
Let p = \I'#-P be the law of {ut{Xt))te[o,i] under P G P{Q). By contraction of the relative 
entropy (an easy consequence of fl271) ). we have H{p\r) = /7(\l'#P|\I'#/2) < H[P\R) < oo. 
This is the second assumption of Lemma [4.31 and this lemma tells us that {ut{Xt))te[o,i] 
is a nice P-semimartingale, i.e. u G dom£^. □ 

Lemma 4.8. Let P G P(f2) be specified by f[T^ with miV > — oo and fo,gi > 0. Then, 
for H{P\R) < oo, it is sufficient that /q log^(/o) dm < oo and gf log^((7i) dm < oo 
for some p > 1. 

Proof. Since m and R are bounded positive measures, only the large values of the functions 
are important as regards integrability issues. As exp(— J^^ ^ rft) is bounded, all we 
have to show is that if two nonnegative functions F = /o(Xo) and G = gi{Xi) satisfy 
/ F^ log^(F) dR <oo and / log^(G) dR < oo with p > 1, then / FGlog+(FG') dR < 
oo. 

For all x,y > 0, we have xy < log_,_ x when y < x log_,_ x and in the alternate case when 
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y > xlog_,_a;, we see that for any < g < 1 and y > Uq large enough, x < y{log_^_y)^'^. 
Hence, 

xy < log+ X + ?/^(log+ yy^, Vx > 0, ?/ > y^. 
Now, for F, G large enough we have 

FG'log+(FG) < (Flog+F)G + FGlog+G 

^ p^^2i ^ , (Flog+F)^ , (Glog+G)2 
< F log , F + G log , G + - — g ,rn, + 



log^i^log+F) log;(Glog+G) 
< 2F^ logl'" F + 2G^ log+"^ G 

which completes the proof of the lemma. □ 

Let x(a) be a Young function, then 

7/(a) :=x(|a|log+|a|) and 7s(&) := log+ |&|) (29) 

are also Young functions. Clearly, a61og_,_(a6) < (a log_,_ a)6 + a(6 log_^ 6) < 2[7j(a) +7g(6)] 
for any large enough positive numbers a, b. Therefore, if / G L"^^ and g G L"^^ , then 
fg G LlogL. 

Gathering our last results leads us to the following statement. 

Theorem 4.9. Let P be the generalized h-process given at (IT^ with iniV > — oo and 

the functions /o and gi such that one of the following conditions is satisfied: 

(i) fo G L'^f (m) and gi G L'^'-'im) where 7j and 7^ satisfy 
(a) /q log^(/o) dm < 00 and g\ log+((7i) dm < 00 for some p > 1. 

Then, H{P\R) < 00 and any function u G dom£-^ which is in the class Ur is also in the 
extended domain dom associated with P. 

As particular cases of condition (i) above, we have /o G L'^{m),gi G L log L(m) and 
fo G L\ogL{m), gi G L°°{m). Condition (ii) is a slight improvement of condition (i) with 
Xix) = x^. 

The stochastic derivative of P. Let us start saying some words about the carre du 
champ operator of a Markov process P. It is a general result of the theory of stochastic 
processes that the product of two real semimartingales is still a semimartingale. More 
precisely, if Y and Z are semimartingales, then 



YZ = YZo + j Y_dZ + J Z_dY + [Y, Z] 



where [Y, Z\ is the limit along refining finite partitions of the time interval by means of 
stopping times: < Ti < • ■ ■ < = 1, of the cross variation X]i(^r,+iAt-'i^T,At)(^T,+iAt- 
^TiAt)- It is a remarkable result that [Y, Z] is again a semimartingale. Its compensator is 
denoted by {Y, Z) , this means that 



Y.Z 



[Y,Z] = {Y,Z) + M' 

where (Y, Z) is a predictable bounded variation process and M^'^ is a local martingale. 
Nevertheless, the product of two nice semimartingales might not be nice anymore. Let Y 
and Z be nice. Clearly, the stochastic integrals J Y_dZ and J Z^dY are nice so that Y Z 
is nice if and only if (F, Z) is absolutely continuous. 
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Definition 4.10 (Carre du champ operator). Let u and v he two measurable real func- 
tions on [0, 1] X X . Going hack to the canonical process X on fi, suppose that the pro- 
cesses u{X) = {ut{Xt))t£[o,i] and v{X) = (t>t(Xi))tg[o,i] ^^'^e P -semimartingales such that 
{u{X),v{X)) is absolutely continuous P-a.s. Then, we say that the couple of functions 
{u,v) is in the domain domF^ of the carre du champ operator which is defined by 

d{u{X),v{X))t =: T^{u,v){t,Xt-)dt, P-a.s. 

This identity determines the function {t,x) G [0, 1] x A" i— )■ T^{u,v){t,x) G M, dtPt{dx)- 
almost everywhere. 

As a direct consequence of this definition, we obtain the following result which is often 
used as a definition of T^. 

Proposition 4.11. Let u and v be two continuous functions on [0, 1] x A" such that u,v 
and their product uv belong to dom£^. Then, {u,v) G domF^ and 

T^{u, v) = C^{uv) — uC^v — vC^u. 

Proof. We denote Ut = u(t,Xt) and Vt = v(t,Xt). By hypothesis, we have dUt = 
C^u{t, Xt) dt + c/M«, dVt = C^v{t, Xt) dt + dM^ and d{UV)t = C^{uv){t, Xt) dt + dM™ 
where M stands for any local i?-martingale. Therefore, 

d[U,V]t = d{UV)t-UtdVt-VtdUt 

= [C^{uv) - uC^{v) - vC^{u)]{t,Xt-) dt + dMt 

with dMt = dM^^-UtdM^-VidMl". Hence, d{U, V)t = [CP{uv)-uC^v-vC^u]{t,Xt-) dt, 
which is the announced result. □ 

There are no tractable general conditions on P which imply that {u{X),v{X)) is ab- 
solutely continuous P-a.s. whenever M,f G dom£^. Counterexamples are known, see 
[Mok89j : u,v E dom£^ doesn't imply in general that {u,v) G domF^. Some additional 
assumptions are needed. 

Theorem 4.12. Let the h-process P and the function gt{x) be defined by fll9l) and fll4l) . 

Let the hypotheses of Theorem \3.^J\ and Proposition^^ be satisfied: 

(i) R E P(f^) is a stationary Markov process with invariant law m = Rt E ^{X) for all 
tG [0,1]; 

(a) •y is a Young function which satisfies f|T6|) and •y* is its convex conjugate; 
(Hi) V is a measurable function on [0, 1] x which is bounded below and is such that 

{7*(^)i^ £ [0, 1]} is uniformly integrable in L^{m); 
(iv) gi is a nonnegative function on X in L'^{m). 



We also assume that /o and gi satisfy the hypotheses of Theorem to insure that 
H{P\R) < oo. 

Then, Ur C domC^ C domL-^ and for all u G Ur which satisfies for almost all t G [0, 1) 
and m-almost all x, 



sup Er{\C Us\^ \ Xt = x) < oo, for some ho > and p > I, (30) 

se[t,t+ho] 



we have 
and 



R 



{g, u) G domF 

C''u{t,x) = L''u{t,x) = L'^u{t,x) + ^'^(9,u)it,x) ^ dtPt{dx)-a.e. 



9t[x) 

where no division by zero occurs since gt > 0, Pt-a.s. 
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Proof. Let u be in Ur, then we know by Theorems 12.91 and 14.91 that 

M e dom£^ C doniL^. (31) 
With ( l23ll we see that for all < t < t + < 1 and Pt-almost all x, 

Ep{ut+h{Xt+h) - ut{x) \Xt = x) 

= En ( gtixy^lut+hiXt+h) - Ut{x)] exp I - / Vr{Xr) dr ) gt+h{Xt+h) \Xt = x\ 
\ V J[t,t+h] J J 

with gt{x) > 0, Pt{dx)-a..s. We write for simplicity Us{Xs) = Ug, gs{Xs) = Gs, K = Vg^Xg), 
D^Ut = Ut+h{Xt+h)-Ut{x), D^Gt = gt+h{Xt+h)-gt{x) and D^F^ = J^^^^^^j dr. The 

inner term in the right-hand side expectation is 

gt{x)''D^Ute~'''^^Gt^n 
= D^Ut{l + [e-^"^' - 1])(1 + D'^Gt/gtix)) 

= D'Vt + D'VtD'^Gt/gtix) + [e"^'^' - l][D''Ut + D'^UtD'^Gt/gtix)]. (32) 

As it is assumed that u G dom£-^, (t/r)rG[o,i] is a i?-semimartingale. Since its sample 

paths are in Dr, they are bounded _R-a.s. and the sequence of stopping times inf{r G 

[0, l];\Ur\+Gr>k} converges i?-a.s. to infinity. Therefore, we can assume without loss 

of generality that U and G are bounded without introducing integration times. 

The contribution of the first term D^Ut of ( l32l) is well understood. Since u G domL^, we 

have 

hm rElD'^Ut = L%(t, x) (33) 
HO h 

where we denote = Er{- \ Xt = x) for simplicity. 

Let us control, the last term of ( l32l) . As G and U can be assumed to be bounded, DUt 
and DGtDUt are also bounded. Hence, DUt + E>GtDUt/gt{x) is bounded and by right 
continuity of the sample paths, it tends to zero i?-a.s.. By dominated convergence, we 
obtain 

lim E%j{DUt + DGtDUt/ gt{x)) = 0. 
On the other hand, le"^"^* - 1| = lie-^"^' - l\/{-D^Ft)\ \D^Ft\ < e^°^| J^^^^^j K- dr\ and 

Ell* (^[e-^'^' - 1]) < c,,^,Eli* (1 1^ ^^^^ K dr^ < c,,,,El^ ^^^^ 7*(K) 

',A„^ / i?K7*(K) dr < c,*,A„ sup i?^7*(V;) < oo. 

IT' J\t,t+h] refo,!] 



It follows with Holder's inequality that 

[e-^'^' - l][Z}'^f/, + D^UtD^Gt/gt{x)] 



lim -E^ 

/i40 h 



Let us look at D^UtD^Gt coming from the second term of fl32l) . By means of basic 
stochastic calculus we arrive at 

DGtDUt = [ (Gr - Gt) dUr + [ (Ur - Ut) dGr + [G, U]t+h - [G, U]t. 

J[t,t+h] J[t,t+h] 
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With dUr = C^Ur dr + dM]^ and dGr = C^Qr dr + dM^ = VrGr dr + dM^ where we rehed 
on Theorem 13.31 in last equahty, taking the expectation leads us to 

EUDGtDUt) 

= El! (G, - Gt) C%r dr + Elf {Ur - Ut) VrGr dr + U]t+h - [G, U]t) . 

J[t,t+h] J[t,t+h] V ■' 

" V ' ^ V ' Oh 

Ah Bh 

Let us control A^, and Gh. By Holder's inequality with 1/p + 1/g and g > 1, 

'[t,t+h] / V J[t,t+h] 



Ah < (e^, [ \Gr - Gtl'dr) ( E% [ {C^rl'' dr] 

V J\t,t+h] / V J\t,t+h] / 



But E^J^_^^^^ \Gr — Gtl'^ dr = o{h) since {G^; r G [0, 1]} is bounded and G is right con- 
tinuous. We also obtain, Jj^^^^j \C^Ur\^ dr = f^_^ ^^^-^ Ef^\C^Ur\^ dr = 0{h), under the 

condition that ^ holds. It follows that Ah = o{hf/'iO{hf/'P = o{h). 
Let us control Bh- We can take U bounded and we already know by Lemma 13.21 that 
{VtGt;t G [0,1]} is uniformly integrable. Since U is right continuous, it follows that 
Bh = o{h). 

We know by f l3T|) that the limit 

lim l-EUD'^Ut + D^UtD'^Gt/gtix) 
HO h ^ 

+ [e-^"^* - l][D''Ut + D^UtD^Gt/9t{x)]} =: L''u{t,x) 
exists. We have also shown (155]) and fl34p which imply that, dtPt{dx)-a.e. : 

L^u(t,x) = L^u(t,x) + gAx)^^ lim ]-Gh 

hio h 

= L''u{t,x) + gtix)-Hun]-EU[G,U]t+h - [G,U]t) 

= LMt, ^) + 9t{x)-' lim jEUiG, U)t+h - {G, U)t) 

and in particular that the limit lim/jj^o j^^iiiiCr, U)t+h — {G, U)t) exists. Since this is true 
for all t and x, this shows that {g^u) belongs to the domain of F^. We conclude noticing 
that by definition lim^^^o U)t+h - {G, U)t) = T^ig, u){t, x). □ 

We note for future use the following result. 



Corollary 4.13. Under the assumptions of Theorem ]^. 1S\ we have 

T^{g,u){t,x) = lim ^ER([gt+h{Xt+h)-gt{x)][ut+h{Xt+h)-Ut{x)]\Xt = x), dtni{dx)-a.e. 
/i— j-oo ri \ / 

(35) 

The product ug is in dom£^ and r^{g,u) = C^{gu) — gC^u — uC^g. 

Proof. The identity ( 135|) has been proved during the previous proof of Theorem l4.12[ Next 
assertion follows from D{GU) = DGDU + UDG + GDU and the convergences which are 
implied by m,^' G dom£^ and {g,u) G domF^. □ 
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Remark 4.14. Let us also remark that applying Lemma 14.31 to the quadratic variation 
under the assumption H{P\R) < oo we see that d{u{X))f <^ dt, i?-a.s. implies 
that d{u{X))^ < dt, P-a.s. It follows that 

domF^ C domF^ 

in the sense that we consider (itPt((ia;)-a.e.-defined functions instead of dtm{dx)-SL.e.- 
defined functions. 

In the special case when X is continuous i?-a.s., we also have {u{X))^ = {u{X))^, P-a.s., 
which implies that T^{u,v)(t,x) = T^{u,v)(t,x), dtPt{dx)-a.e., {u,v) G domF^. 

5. Continuous diffusion processes on 

In this section we examplify the previous abstract results with simple continuous dif- 
fusion processes on W^. 

The reference process R. The reference process R is the law of a Markov continuous 
diffusion process on the state space A* = M°' which admits an invariant probability measure 
m. To fix the ideas, we assume in the whole section that it is the solution of the stochastic 
differential equation (SDE) 

Xt = Xo+ [ b{X.,) ds+ [ a{Xs) dWs, t G [0, 1] 

J[0,t] J[0,t] 

where is a M'^-valued Wiener process, 6 : R'^ — > M'^ and a : M*^ — )■ M^xd are locally Lip- 
schitz functions which are respectively vector-valued and matrix-valued. We also assume 
that P-a.s., X doesn't explode on the time interval [0, 1]. 

Result 5.1. Under these hypotheses on R, it is known that R is the unique solution of the 
martingale problem MP(£, C;/io) in the sense of Definition \2.4 with the initial measure 
fj,o = rn and the generator given for all u E C = C^'^([0, 1] x M'^) by 

d 1 ^ 

C%{t,x) = dtu(t,x) + ^bi{x)d^^u(t,x) + 2 aij{x)d^^d^.u(t,x) 

where iaij)i<ij<d = a := aa* G Md^d- 

We denote this martingale problem MP(6, a;m). 

Extended gradients. We introduce the notion of extended gradient. Let P be a solution 
to the martingale problem MP(6'^, a; Pq), for some drift vector field b^ : [0, 1] x R'^ — )■ R*^. 
A simple computation based on Proposition 14.111 gives us 

r^((^,t;)(t,x) = V'^t{x)-a{x)Vv{x), dtPt{dx)-a.e., ip G Cl'^{[0,l]xW^),v G ^^(R'^). (36) 

One proves the Cauchy-Schwarz type inequality 

( [ d{A,B)r^ < [ d{A,A)r f d{B,B)r, 0<s<t<l 

\Jls,t] J J[s,t] J[s,t] 

with the usual discriminent argument. Let us take u,v in C^(R'^) and ip a measurable 
function on [0,1] x X such that {ip,u) and are in domP^ and such that r^{v — 

u,v — u) = 0. Then, the above Cauchy-Schwarz inequality implies that r^{tljt,u){t,x) = 
r^lil^t, v){t, x), dtPt{dx)-a..e. Consequently, the linear operator u T^{ip, u) only depends 

on the equivalence class defined by m ~ w 4=> T^{v — u,v — u) = 0,dtPt{dx)-a..e. 
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a ■ V(f — u) = 0,dtPt{dx)-a..e., and it follows that there exists some vector field /3 on 
[0, 1] X R"' such that T^lip,-) is represented by 

r^{tlj,v){t,x) = Pt{x) ■ a{x)Vv{x), dtPt{dx)-8i.e., v e C'^{R'^). (37) 

Moreover, up to dtPt{dx)-a..e. equality, there is a unique such /3 with its values in the 
range of a. 

Comparing (!36l) and (1371) . it is natural to introduce the following definition. 

Definition 5.2 (Extended gradient). Let ip be a measurable function on [0, 1] x M'^ such 
that for all u G C2(M^), {ip, u) is in domF^. The unique vector field /3 which satisfies 
fl37|) and f3t{x) G Range a(a;) up to dtPt{dx)-a.e. equality is denoted by (3 = V^ip and it is 
called the P -extended gradient ofip. 

When no confusion can occur, we simply drop P and write V^ip = Vip. 

It is clear with our previous discussion that for any u G C^(M'^), Vu is the orthogo- 
nal projection of Vu on the range of the diffusion matrix a. In particular, Vm = Vu, 
dtPt{dx)-a..e., when a{x) is invertible for all x G M'^. 

The martingale problem which is solved by P. Now we consider the generalized 
/i-process P. We are going to see that P solves a martingale problem MP(6 + a/3, a) and 
that the additional drift /3 has the special form 

P = V^t/;, P-a.s. 

with ip = \ogg, i.e. 



i){t,x) := log Er 



exp f - / Vs{Xs) ds)gi{Xi) \ Xt = x 
^ JH.l] ^ 



dtPt{dx)-a.e. (38) 



which is well-defined dtPt{dx)-a..e. since g{t, x) > 0, dtPt{dx)-a,.e., but might not be defined 
dtm{dx)-a..e. in general. 

Lemma 5.3. Assume that R satisfies the hypotheses of Result \5.1\ and P defined by f|T9l) 
satisfies the hypotheses of Theorem \4.1S\ Then, for all u G Ur which verifies fl30|) . {ip,u) 
is in domF^ and 

— {t, x) = T^{tlj,u){t,x), dtPt{dx)-a.e. 
9 

Proof. Let us denote Zt = dP^Q^t^/dR^Q^ty As Z admits a continuous version and Zt = 
/o(Xo) exp ^— JjQ^j Vs{Xs) ds^ Gt with Gt := gt{Xt), G also admits a continuous version. 
Applying Ito's formula to the continuous process ipt{Xt) = logG^, we obtain 



dGt ld{G)t p 
d^tiXt) = - 2~CP~ ^^^^ 

We deduce from this with Theorem 14.121 that for any u G Ur which verifies fl30|) , 
d{iP{X),u{X))t = d{G,u{X))t/Gt = [T^{g,u)/g]{t,Xt) dt, P-a.s. This completes the 
proof of the lemma. □ 

Theorem 5.4. Assume that R satisfies the hypotheses of Result I5.il and let P be the 

generalized h-process which is defined by f|T9l) . Assume also that fo,gi and V satisty the 
hypotheses of Theorem 4-12. 

Then P is the unique solution in {Q G P(^^); H{Q\R) < oo} of 

P G MP{b + aV^i/j,a;Po) 



28 CHRISTIAN LEONARD 

with Pq = foQQ m and where the function 
ip{t,x) := log g{t,x) = log Er 



exp(- / VsiXs)ds]gliX^) \ Xt = x 



dtPt{dx)-a.e. 



is defined by (iMj) and fl38 



Proof. Choosing Ur = C]'^([0, 1] x W^) in Theorem 14.121 the assumption ( !30|) holds true 
for all u e C]'2([0, 1] x R'^). The result now follows from Theorem 14.121 and Lemma 15. 3[ 
The assertion Pq = f^gQ m is fl22]) . 

The uniqueness is implied by a general result of Girsanov's theory since R is the unique 
solution to its own martingale problem and H{P\R) < oo. For an entropic point of view 
under the present requirement that H{P\R) < oo, see |Leob] . Otherwise, when P R 
is only assumed this is a standard result of Girsanov's theory, see |JS87] . □ 

Kolmogorov diffusion process. We illustrate this theorem by means of a diffusion 
process which plays an important role in the area of functional equalities connected with 
the concentration of measure phenomenon |Bak94^ Roy99 , ILedOll IVilOQj . 
The Kolmogorov diffusion process is the unique solution of the SDE 

dXt = -VU{Xt) dt + dWt 

where U is a C^-differentiable function on M*^ such that Zu := J^d e~^^*^^^ da: < oo. This 
SDE admits the Boltzmann-Gibbs probability measure 

m^idx) := Z^'e-^^^'^Ux 

as a reversing measure. We take this reversible Kolmorov diffusion as the reference process 
R. Hence, the initial law is Rq = mF and 

R G MP(-Vf/,Id). 

The generalized /i-process to be considered here is P specified by f[T^ with the assumptions 
of Theorem 15. 4[ This theorem tells us that 

P e MP(-Vf/ + V^^/',Id). 

In the special case when the potential V is zero, we have for all < t < 1, 

gt{x) = ER{g,{X{) \ X^ = x) = [27r(l - t)]-^/^ gM exp (|^^) dV- 

Therefore, g G C°°([0, 1) x W^) and gt is positive for all < t < 1. It follows with 

iJt{x) = logER{g,{X,) \Xt = x), te [0, l),xe 
that V^ipt = V-J/^t and 

P eMF{-V[U -tlj]M) 
and with Theorem 13. 3 1 we see that ip is a classical solution of the Hamilton- Jacobi-Bellman 
(HJB) equation 

£^^(t,x) + ||V^t(x)p = 0, t G [0,l),x G M"^ 

limt-^i tpt{x) := ^i{x) = loggi{x), t = l,x E {gi>0} 



where 



C%{t,x) = (dt-VU{x) - V + ^A^u{t,x). 
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Let US go back to the general case when V is not constant. The positivity improving 
property of the heat kernel implies that tpt is well-defined for all t G [0, 1). But it might 
not be smooth enough to be a classical solution of the HJB equation: 

C^t{j{t,x) + ||VV't(x)p - V{t,x) = 0, te [0,1), X e M'^ 

limt^i ipt{x) := tpi^x) = \oggi{x), t = l,x e {gi > 0} 

Because of its semigroup representation (1551) . ip is a continuous viscosity solution of this 
equation, see |FS93t Thm II. 5.1] for instance. 

6. CONTINUOUS-TIME MARKOV CHAINS 

In this section we examplify our results with simple Markov jump processes on a count- 
able discrete space X which are analogous to the Kolmogorov diffusion processes. The 
set of paths is Q = D{[0,1], X). 

The reference process R. Since X is a. countable discrete space, every function is 
measurable and continuous. Let B{X) denote the space of all real bounded functions on 
X. The first ingredient is a Markov generator 

u{y) - u{x)] J°(x; dy), u E B{X) (40) 

where J" is a kernel of positive measures on X such that J°(x; {x}) = for all x E X and 

(i) J°(x; X) < oo, for all x e X; 

(ii) J° induces an irreducible process in the sense that J^{x] A") > for allx E X and for 
any couple of distinct states (x, y), there exists a finite chain x = Zi, Z2, ■ ■ ■ , Zn = y 
such that J{zi; {-Zj+i}) > for all i; 

(iii) J° satisfies the detailed balance condition 

m\dx)j\x; dy) = m\dy)j\y; dx) (41) 

for some nonnegative measure m° on X (possibly with an infinite mass). 

We say that Q G P(^^) solves the martingale problem MP{K) associated with the pre- 
dictable jump kernel K = K{t, Xp^i); dy), if 

u{t, Xt) - u{0, Xo) - / ds f [u{s, y) - u{s, X,-)] K{s, X[o,,); dy), te [0, 1] 

J[o,t] Jx 

is a local Q-martingale for a large class of functions u. 

Under the assumption (i), there is a unique law G P(^^) which solves the martingale 
problem with a prescribed initial law and the Markov generator fl40l) : R^ G MP(J°). 
Under the assumption (iii), the measure mP is its invariant measure which is unique (up 
to scalar multiplication) under the irreducibility assumption (ii). 

The second ingredient is a potential U on X such that Zu := J^e~'^'^ dm < +oo. The 
reference process R is the law of the Markov jump process with generator 

C^u{x) := / [u{y) — u{x)] J {x; dy) , ueB{X) where 
Jx 

J{x;dy) := exp(-[f/(y) - f/(a;)]) jO(a;; dy) 
which is well defined for all u G B{X) provided that 



I e-^^y^ J°(x; dy) < +oo, Vx G X, 
Jx 



30 CHRISTIAN LEONARD 

as this last integrability assumption implies that 

J{x] X) < oo, Vs G X. (42) 

It is easily seen that the Boltzmann-Gibbs probability measure 

m^{dx) := Z^'e-^"^''^m\dx) 

is the reversing measure of the jump process R since the detailed balance conditions are 
satisfied. Indeed, 

m^{dx)J{x; dy) = e-2t^We-[^(?^)-^(")] m°(c/a;) J°(x; dy) 

= e-[^(-)+^(2^)] m°(c/x) J°(x; dy) = e-[^(")+^(?')l w!'{dy)J%y] dx) = m^{dy)J{y; dx) 
where (jH]) has been used at the last but one equality. Therefore, 

R G MP(J;m^). 

Moreover, it is the unique solution of this martingale problem. Indeed, thanks to (142|) it 
is possible to build a unique strong solution on some auxiliary probability space: a com- 
bination of a discrete-time Markov chain with transition probabilities J(x; dy) / J{x; X) 
and independent exponential clocks with frequencies J(x; X), x G X . 

This reference law is sometimes called a Metropolis dynamics on the set A". It is useful 
for estimating when the very high cardinality of X prevents us from computing the 
normalizing constant Zu. 

The martingale problem which is solved by P. Now we consider the /i-process P. 
Applying Theorem \4.V2\ we need to compute r^{g,u)/g for a large class of functions 
u G Ur. We choose this class to be B{X) for the following reasons. On one hand, we 
can see that B{X) C Ur because with fH2|) it is clear that B{X) C dom£^ and for all 
u G B{X) and a > 0, Jjg ^^^■^exp{a[u{y)—u{Xt-)]) dtJ{Xt-; dy) < oo. On the other hand, 
we also see immediately with (H2l) that (130|1 holds for any bounded function u. 

Theorem 6.1. Let R G MP(J;m'^) be as above, P be the h-process specified at fll9p and 
assume also that fo,gi and V satisty the hypotheses of Theorem \4.1^^ 

Then P is the unique solution in {Q G P{Q); H{Q\R) < oo} of MP{J^; Pq) with Pq = 
fogom and 

9t{y) 
9t{x) 



J^{t,x;dy) = exp fV't(y) - ^ptix)] J{,x]dy) = ^^-^ J {x; dy) , dtPt{dx)-a.e 
where the function 
ijj{t,x) := log g{t,x) = log Er 



exp ^ Vs{Xs) ds^ gi{X,) \ Xt 



dtPt{dx)-a.e. 



is still defined by (fT4l) and (!38l) . 

Proof Corollary SH tells us that for all u G B{X), T^ig,u) = C^{ug) - uC^g - gC^u. 
Hence, T^{g, u){t,x) = J^^luiy) — u{x)][gt{y) — gt{x)] J{x; dy), dtm{dx) and 

^^^^{t,x) = [Hy)-u{x)](^-l) J{x-dy) 
9 Jx \9t[X) J 

f [u{y) - u{x)] (e^*(f)-V'*W _ i) j(x; dy), dtPt{x)-a.e. 

X 
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We conclude with Theorem 14.121 that P solves the announced martingale problem. The 
uniqueness statement follows from the general Girsanov theory: because P <^ i?, it is 
inherited from the fact that R is the unique solution of its martingale problem. □ 

As with the continuous diffusion processes, we see that some gradient of ip is involved 
in the shift from the dynamics of R to the dynamics of the /i-process P. Indeed, denoting 

Du{x; y) := u{y) — u{x) 

the discrete gradient of u at x, we have 

J^(x; dy) = exp [DiJtix; y)) J(x; dy). 



With Theorem 13.31 we know that C^g = Vg. If g is time-differentiable and positive on 
[0, 1) X X, we deduce that is a. classical solution of the following integro-differential HJB 
equation 

C^'ilj{t,x)+ / 9{D'iljtix;y))J{x;dy)-V{t,x) = 

where 9{a) := — a — 1 and is the generator whose value on any t-differentiable 
bounded function u is 

C^u{t, x) = dtu{t, x) + / Dut{x;y) J{x; dy). 

Jx 

In the general case when g might not be time-differentiable and positive on [0, 1) x X, the 
semigroup representation of ip implies that ip is the unique continuous viscosity solution 
of the HJB equation 

C^^{t, x) + e{Dtpt{x; y)) J{x; dy) - V{t, x) = 0, te[0,l),xe R'^ 



limtii V'j(x) := ipiix) = \oggi{x) 



t = l,x e {gi> 0} . 
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